How to: Close all open windows of all workbooks.

Solution:

Press and hold down SHIFT while selecting the 'File' menu and selecting 'Close All'.